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SUMMARY 
Logical  design  of  an  optimal  network  by  integer  linear  pro- 
gramming is  formulated.   Various  restrictions  on  a  network  are  easily 
incorporated  and  how  to  incorporate  them  is  discussed.   The  number  of 
gates,  the  number  of  levels  or  one  of  other  parameters  of  a  network 
can  be  minimized.   A  network  of  multiple  outputs  is  also  easily 
treated.   The  formulation  is  based  on  threshold  gates,  since  threshold 
gates  are  the  generalization  of  conventional  switiching  gates,  such  as 
AND,  OR  and  NAND  gates.   The  formulation  of  designing  a  network  of  con- 
ventional gates  can  be  easily  derived  from  this  general  formulation. 
As  an  example,  design  of  an  optimal  network  with  NOR  gates  is  dis- 
cussed in  some  detail.   Computer  programs  to  design  optimal  networks 
with  NOR  gates  for  all  Boolean  functions  of  three  variables  are  run. 
Computation  time  is  much  less  than  the  exhaustive  method  of  finding 
optimal  networks.   The  design  by  integer  linear  programming  is  found 
computationally  feasible. 


1.   Introduction 

Logical  design  of  an  "optimal"  digital  network  has  been  one  of 
the  most  important  objectives  of  switching  theory.   Optimality  of  a 
network  is  the  minimization  of  the  number  of  gates,  connections, 
levels  or  whatever  the  designer  wants  to  minimize,  under  network  re- 
strictions such  as  fan-ins  and  fan-outs  restrictions.   But  the  design 
of  an  optimal  network  has  remained  an  unsolved  problem  since  it  is 
difficult  to  formulate  the  concept  of  optimization  within  the  frame- 
work of  Boolean  algebra  which  has  been  a  major  mathematical  tool  in 
switching  theory. 

With  the  advent  of  integrated  circuits  and  large  scale  integra- 
tion the  traditional  optimality  of  a  network  has  been  further  com- 
pounded by  many  engineering  considerations  and  restrictions.   Thus 
within  the  field  of  design  automation  there  is  a  great  need  for  a 
flexible  optimization  technique  which  will  allow  the  designer  to 
specify  a  variety  of  composite  optimization  criteria.   Such  a  tech- 
nique would  not  only  reduce  the  cost  of  digital  networks  but  also 
reduce  the  complexity  and  increase  the  reliability. 

In  this  paper,  the  problem  is  approached  by  means  of  integer 
linear  programming.   A  preliminary  computational  result  is  also 
outlined. 

The  integer  linear  programming  approach  is  flexible  to  incorporate 


a  wide  variety  of  constraints*  which  are  to  be  imposed  on  a  network 
to  be  designed  and  to  use  any  of  various  types  of  criteria.   Another 
important  feature  of  the  approach  is  thst  incompletely  specified 
functions  can  be  treated  with  no  essential  difference  from  the  case 
of  completely  specified  functions,  contrary  to  the  Boolean  algebraic 
manipulation.   A  multiple  output  network  can  also  be  handled  without 
any  major  modification. 

An  integer  linear  programming  problem  in  general  is  stated  as 
follows : 

minimize     c"  y 

subject  to     Ay  >  b 

(1.1) 
y  >  o  , 

where  a  subset  of  the  variables  y  are  constrained  to  be  integers, 
c  is  an  N-dimensional  vector,  b"  is  an  M-dimensional  vector  and  y  is 
an  N-dimensional  vector  of  variables.   A  is  an  M  x  N  coefficient 
matrix.   N  is  the  number  of  variables  and  M  is  the  number  of  in- 
equalities. 


In  contrast  to  our  approach,  design  procedures  which  have  been 
known  to  date  have  no  flexibility  in  using  different  constraints. 
For  example,  the  well  known  design  approach  by  the  minimization 
of  a  prime  implicant  expression  leads  to  an  optimum  network  only 
when  the  network  is  to  have  two  levels,   (integer  linear  pro- 
gramming approach  which  is  completely  different  from  ours  in  the 
present  paper  can  be  partly  used1-  J).   Gimpel's  approach^  leads  to  an 
optimal  network  only  when  the  network  is  to  have  three  levels. 


The  above  integer  linear  programing  problem  is  called  all- integer 
integer  linear  programming,  or  mixed-integer  integer  linear  program- 
ming, depending  on  whether  all  the  variables  are  constrained  to  he 
integers  or  not.   Each  variable  may  be  further  specified  to  assume 
general  non-negative  integral  values  or  binary  integral  values  which 
are  1  or  0. 

There  exist  basically  a  few  algorithms  for  integer  linear  pro- 
gramming.  The  cutting  plane  method  by  Gomory         may  be  applied 
to  general  problems  of  both  all-integer  and  mixed- integer  integer 
programmings .   Another  is  the  implicit  enumeration  method  or  the 
branch  and  bound  method( [1]  [7]  [8]  [10]  [l6]  and  possibly  others) 
which  is  particularly  useful  to  the  problem  of  (0,  l) -valued  vari- 
ables.  One  of  the  remarkable  properties  of  integer  programming  is 
the  erratic  convergence  speed  of  a  method,  depending  upon  the  type 
of  a  particular  problem.   A  method  which  can  solve  efficiently  one 
problem  may  not  be  efficient  for  others.   Probably  each  particular 
problem  has  a  suitable  method  and  furthermore  the  method  can  be  modi- 
fied so  that  the  convergence  is  sped  up. 

An  integer  linear  programming  approach  was  first  brought  into 
the  design  of  an  optimum  feed-forward  network  of  threshold  gates  by 
S.  H.  Cameron  in  I96U   .   His  formulation,  however,  needs  an  ex- 
ponentially increasing  number  of  variahles  as  the  number  R  of  gates 

in  a  network  increases,  while  our  approach  uses  the  number  of  vari- 

2  [1*] 

ables  in  the  order  of  R  .   Gameron's  work  was  followed  by  M.  Breuer's 

which  considerably  simplified  the  formulation.   Our  work  which  is 

essentially  the  same  approach  as  Breuer's  but  discussed  more 

comprehensively  various  aspects  of  the  formulation  was  initiated 

independently  (the  first  presentation  was  [20])  and  needed 


slightly  fewer  inequalities  than  Brevier* s.   The  current  paper  is  a 
revision  of [20]  and  will  be  a  basis  of  the  further  application  of 
integer  linear  programming  to  logical  design  which  will  be  published 

i        *   [22] 
elsewhere 

In  the  following,  let  us  first  introduce  our  integer  linear  pro- 
gramming formulation  based  on  a  network  of  threshold  gates.   This  is 
a  general  formulation  because  conventional  switching  gates  such  as 
AND,  OR,  NAND  and  NOR  may  be  considered  as  special  threshold  gates 

with  their  weights  and  thresholds  appropriately  specified.  As  we  will 

[221 
see  in  the  succeeding  paper    ,  this  general  formulation  will  be  a 

basis  of  logical  design  of  more  complicated  networks  such  as  a  network 
where  each  gate  may  be  chosen  from  a  given  set  of  different  types  of 
gates.   (Accordingly  a  resultant  network  consists  of  a  mixture  of  dif- 
ferent types  of  gates.) 

Logical  design  of  an  optimum  network  with  only  NOR  gates  will  be 

discussed  later  as  an  example  of  special  cases  of  the  general  formula- 
tion. 


2.   Definition 

A  threshold  gate  is  defined  as  a  logic  gate  in  which  each  variable 
input  x»  (i  =  1,  2,  . ..,  n)  is  1  or  0  and  for  which  there  exists  a 
set  of  real  numbers,  w  ,  . . . ,  w  ,    called  weights  and  T  called  a  thres- 
hold such  that  the  output  of  the  gate  is; 


n    /  .s 

1  if   S  VnXf       >   T 


£=1 


(2.1)- 


n 


.(J) 


0  if  Z  ^gX-Y1  <  T  -  1,  (j  =  1,  2,  ...,  m)  , 
4=1  *  * 


where  x    =  (x.J   ,  . ..,  x   )  is  the  j-th  input  vector  and  m(<  2  )  is 
the  number  of  input  vectors  which  specify  the  output  values  of  the  gate. 

A  feed-forward  network  is  shown  in  Fig.  1  in  which  all  threshold 
gates  are  arranged  in  a  line  and  each  threshold  gate  in  the  network 
can  receive  the  inputs  only  from  the  external  variables  xn ,    x  ,    ...,   x 
or  from  outputs  of  the  preceding  gates  on  its  left. 


Fig.  1  A  Feed-Forward  Network 


Strictly  speaking,  these  are  called  the  normalized  system  of  in- 
equalities of  a  threshold  gate  and  are  derived  by  taking  into 

[211 
account  the  margin  of  operation.   For  detail,  see 


The  k-th  gate  in  the  network  has  the  weight  vector. 

-4c   /  k   k       v. 
w  =  (w^  w2,  ...,  w*) 

for  the  external  variables  x..,  x_,  . ..,  x  ,  and  a  threshold  T  .   The 
weight  of  interconnection  from  the  i-th  gate  to  the  k-th  gate  (i  <  k) 
is  denoted  by  (X     .   The  output  value  of  the  k-th  gate  for  the  j-th  in- 
put vector  5C        is  denoted  as  P./^  and  determined  by 


P^)  =  1  if  Z    Wo  xi^  +  Z    a..  p(^  >  T 
k       /,!  i  ^    i=1  ik  i   ^ 


(2.2)* 


P,^  =   0  if  Z  w£  x/j)+  Z    a.,  ?[^   <  T,  -  1  . 
k        *  -^  X  /     .__   lk  i   —  k 


Fig.  1  has  R  gates  and  the  final  gate  is  supposed  to  emit  the  output 
whose  value  for  each  input  vector  is  specified  in  advance.  In  other 
words,  if  the  network  is  to  realize  a  switching  function  f,  then 


must  hold  for  each  x   . 


PU)  =  f(^J))  (2.3) 


A  feed-forward  network  is  the  most  general  case  of  a  loop-free 
network.  Henceforth,  we  will  consider  only  feed-forward  networks  with 
respect  to  our  integer  linear  programming  approach. 


* 

0 

Z     is  defined  to  be  o  so  that  the  inequality  expression  (2.2)  may 
i=l 
be  applicable  to  the  first  gate. 
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3-   Description  of  a  feed-forward  network  in  integer  linear  in- 
equalities. 

Here  we  completely  characterize  the  operation  of  a  feed-forward 
network  by  linear  inequalities  with  some  variables  restricted  to  be 
integers.   In  Section  5,  &  synthesis  procedure  of  an  optimal  network 
based  on  this  formulation  will  be  given. 

Let  us  consider  the  k-th  gate  in  a  feed-forward  network  shown  in 

Fig.  1.  A  set  of  new  variables  B.,   is  introduced  in  association  with 

lk 

a.      and  P.   ,  such  that 
lk      1 

(3^  =  a..P^   .  (3.1) 

lk     lk  l 

P.?  may  be  considered  as  the  input  value  from  the  i-th  gate  to  the 

k-th  gate  for  the  j-th  vector.   The  non-linear  relation  (3'l)  will  be 

rewritten  in  linear  inequalities  later.   Then  from  (2.2),  the  output 

of  the  k-th  gate  is  defined  as  follows: 

n   ,   /jN  k-1 


pW  =  1  if  E  w*  .w  +^  P£>>  Tfc 


(3-2) 


However,  these  conditional  relations  can  be  converted  into  a  set  of 
linear  inequalities. 

S  w*  xj^  +  S  p^  -  Tk  >  -  U(l-  .?^\  (3.3A) 

-  Z  w^  x(0j)-  Z  p^  +  T,.  -  1  >  -  UFfr    ,  (3-3B) 

=1 


'JL-L*     *     "i=J 


where  P;«"  is  1  or  0. 
k 

Let  us  assume  that  U  is  a  sufficiently  large  positive  number. 
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Then  if  P)   =  ljthen  (3-3A)  may  be  obviously  rewritten  as 

n      /  .v  k-1  /  .x 
2  W|  xy}  +  Z     p)^'   -  T.  >  0 
§=1  k=l 

and  (3-3B)  becomes  non-restrictive.      Otherwise,    i.e.,    if  P^       =  0, 

( 3- 3B)  may  be  rewritten  as 

n  ,  .x      k-1     /  .n 

-  Z     w o  xV;   -  Z     p)^'  +  T.    -  1  >  0 
Jj   1     /     A         -_-^     ik  k 

and  (3'3A)  becomes  now  non-restrictive.   Therefore  (3«3A)  and  (3-3B) 

together  are  equivalently  used  as  (3*2). 

For  the  above  purpose,  U  is  large  enough  if  it  is  not  smaller 

than  the  conceivable  maximum  of  the  absolute  value  of  the  left  hand  side 

of  each  inequality.   For  (3--3A),  for  instance,  U  satisfies 

m         k-1 
U  >  E  I  wl  !  +  Z   I  «   I  +  1  T.  I 
>1  '     *   '   i=l  '   lk  '      k 

-    k-1    r  .x 
wS  I  +z  !  pu;  I  +  !  T  1        (3.U) 


)i=l        A    1=1    ik 

m   ,„  /  .x  k-1 
E 


p.,    ■ 


wl  4J)+  ^  ^   -k  ■ 


is  satisfactory.  An  upper  bound  of  a  weight  and  a  threshold  of  d  in- 
put threshold  gate  is  known         and  shown  below. 

a+i 
l  w.  i  <  «ff)  2 

d±l  (3-5) 

|  T  I  <  2d(*S)     _ 

Considering  the  k-th  gate  as  a  threshold  gate  with  (n  +  k  -  l)  inputs 
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and  using  (3-^),U    which  is  not  smaller  than 

n+k 
Mn  +  k-  1)   (2j£)  2  (3.6) 

is  sufficient  for  (3. 3A). 

Henceforth  we  will  use  the  letter  U  exclusively  for  this  purpose. 
We  will  use  the  same  U  for  all  inequalities,  though  U  may 
assume  a  different  value  for  a  different  inequality*.  A  lower  hound 
of  U  for  each  inequality  in  this  paper  will  he  ohtained  in  a  similar 
way  as  shown  ahove.   The  derivation  of  each  U  will  he  omitted. 

We  have  introduced  all  necessary  variables  in  the  ahove.  Note 
that  P/.   is  only  variables  which  are  restricted  to  he  integers.  All 
other  variables  are  real  numbers  which  are  possibly  negative.   The 
integer  linear  programming  formulation,  however,  requests  all  vari- 
ables being  non-negative  ( see  ( 1. l) ) .   This  can  be  solved  by  splitting 
each  variable  as  follows: 

k    k+    k- 

vr  ^     -  w^ 

a   =  a  *  -  a  ~ 
ik    ik    lk 

4i]  -  ii)+  -  it ,  (3.7) 


(k  = 

i, 

2, 

•  •  •  f 

R) 

(i  = 

i, 

2, 

•  •  •  f 

k  - 

1) 

(K  = 

if 

2, 

•  •  •  f 

n) 

U  = 

1, 

2, 

• ' '  f 

*), 

For  some  algorithms  of  integer  linear  programming  such  as  Gomory's, 
the  smallest  possible  U  is  preferable  in  order  to  avoid  the  over- 
flow during  the  computation.  The  smallest  values  of  U  are  computed 

T231  \2k~\ 
for  small  numbers  of  inputs        . 

10 


with  constraints 

t§+  >°  >  Jjf  >  o 

a  +  >  o,  a  *  >  o 

ik  —  '   ik  — 

${p+  >  0,  $[p~  >  0   .  (3.8) 

ik   —     ik   — 

For  notational  symplicity,  however,  w«  etc.  will  be  .usually  used  in 

k-f-    k- 
place  of  Wj.  -  wfl  etc.,  unless  otherwise  specified. 

Now  that  output  of  threshold  gate  is  described  by  (3'  3)>  the  non- 
linear equality  ( 3- l)  may  be  expressed  in  linear  inequalities  as 
follows : 

Pik}  -aik  +  U(l  "  Pid))>  (3-9A) 

«ik<  P^}  +U(1  -  Pp}),  (3.9B) 

p[p+  +  p[p~  <  up^  (3.9C) 

ik      ik   —   i  >  v  y    ' 

(i  =  1,  2,    ...,   R  -  1) 

(k  =  i  +  1,  i  +  2,  ...,  R) 
(j  =  1,  2,  ...,  m). 


><3)-  i, 

(3-9)  lead  to 


rt  can  be  easily  seen  that  if  p.  =  1,  then  all  three  inequalities  of 


and  if  P.J'^  =  0,  then 

l      ' 


ik     ik 


v[p  =   0 
ik 


holds,  because  of  P.jj   >  0,  B:jl'"  >  0  and  (3.9.C).   This  is  exactly 
what  (3'l)  implies. 
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The  inequalities  (3-9-C)  for  different  k,  i.e.,  k  =  i  +  1,  i  +  2,  . .., 

R  can  be  packed  into  the  following  single  inequality, 

■p 

Z   (P^)+  +  P^}")  <  UPp\  (3-10) 

k=1+1  (i=  1,  2,    ...,  R-l) 

( J  -  1*  2,  •  •  •  ,m  ), 

reducing  the  number  of  inequalities.   This  is  possible  because  when 

the  output  of  the  i-th  gate  F.^  is  0,  P:J'  =  0  for  k  =  i  +  1,  i  +  2, 

i         ik 

.  . . ,  R  must  follow. 

(3.3)  and  (3-9)  together  characterize  the  entire  feed- forward 

network  except  the  last  gate  (i.e.,  the  Rth  gate).   The  output  of  the 

last  gate  is  specified  by  (2.3).   By  specifying  the  value  of  FlJ  in 

(3' 3)  according  to  (2.3)^  the  inequalities  for  the  R-th  gate 

R-l 


t   w^xy^   +  Z  p?<P  -  T^  >  0    for   f(x^)  =  1 
*=1  ll  i=l  lR     R  " 

(3.H) 

-Z  w?  x^  -  Z  $\y   +  T™  -  1  >  0   for  f(x^)  =  0  , 
$=1  *     *    i=l 

j  =  1,  2,  . ..,  m  , 

are  obtained. 

In  conclusion,  a  feed-forward  network  as  shown  in  Fig.  1  is  de- 
scribed by  the  following  inequalities  with  part  of  the  variables  con- 
strained to  be  integers. 

For  the  k-th  gate,  k  =  1,  2,  ...,  R-l, 

Z  w*  x(/}  ^  ?[$   -  T  >  -  U(1-P^J)) 
i=l  *  ^   i=i  lk    k  -      k 

-Z  w*  x^  -2  3^  +  Tv  -1  >  -  UP[J)  (3.12) 

( j  =  1,  2,  . . . ,  m) 

12 


p[p  <ct..  +  u(i  -  p^) 

lk  —  lk         1  ' 

<t$   +U(1-  P^)  ,  (3.J3)* 


a.. 

ik 

(i  =  1,  2,   ...,   k  -  1) 

(j  =  1,  2,    ...,  m)  , 

•p 

E   ^ki)+  +  Pki)_)<upkJ).  (3.1*0 

i=k+l         K1       K 

(j*  =  1,  2,    ...,   m)  4 

For  the  R-th  gate, 

S  w|  x[j)  +  2  |3^  -  T_  >  0   for  f(x^)  =  1 
1=1        '  i=l  1K     K 

n  R— 1 

-2  w/  x^  -  2  B^  +  L  -  1  >  0   for  f(^d))  =  0  , 
1=1  i=l  1K     K 


(3.15) 


(j  =  1,  2,  ...,  m)  , 


PiR}  -^iR4  U(l  "  PiJ)) 


aiR 


<P^+U(1-  Pp})  ,  (3.16) 

(i  =  1,  2,  ...,  R  -  1) 

(j  =  1,  2,  ...,  m). 

k  ( i) 
Here  w.,  B.   ,  a.    ,   where  k  =  1,  2,    . . . ,   R,  i  =  1,  2,  ...,  k-1, 

'V   IK      IK 

-t  =  1,  2,    ...,    n,  and  j  =  1,  2,  . ..,  m,  are  real  numbers  which  are 

k    k+   k- 
actually  represented  as  w »  =  w „  -  w»  and  so  forth.   The  variables 

P.   are  integers  whose  values  are  0  or  1**. 
l 


*    The  inequality  for  k=l  is  not  needed  in  (3*13) 

**   Inequalities  P.   <  1  must  be  added  when  Gomoryts  algorithm  is 

applied.   In  case  of  the  implicit  enumeration  method,  these  are 

omitted. 
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The  numbers  of  variables  are 

m(R  -  l)        (integer  variables) 
plus  n  R  +  (l  +  m)   ^  ~  '      (real  variables)         (3.17) 

The  number  of  inequalities  is 

m  +  3m(R  -  l)  +  mR(R  -  l)  ,  (3.18) 

where 

R:     the  number  of  gates  in  a  net-work 

m:     the  number  of  specified  vectors 

n:     the  number  of  external  input  variables. 


Ik 


k.      Incorporation  of  network  restrictions. 

Some  of  the  restrictions  imposed  on  a  network  can  be  taken 
into  account  by  simply  adding  inequalities.  Ease  of  incorporation  of 
restrictions  is  one  of  the  characteristics  of  the  integer  linear  pro- 
gramming approach. 

[19] 
First  let  us  discuss  the  restriction  on  input  tolerance    5  i.e., 

the  maximum  permissible  deviation  of  gate  parameters'  such  as  weights, 
a  threshold  and  signal  magnitudes.   The  input  tolerance  ?  for  a 
threshold  gate  can  be  written  as  follows  under  appropriate  assump- 
tions 

min  !   E  w|  x^}  +  "  ?[£    ' 

B=-J fei i=l (k.l) 

n 

Under  other  assumption  a  different  equation  may  result,  as  discussed 

[211 
in    .   One  of  the  other  cases  will  be  given  in  Section  6  in  con- 
junction with  majority  gate  expression. 

When  a  certain  objective  function  is  incorporated,  it  is  always 
satisfied*  by  an  optimum  solution  that 

min  I   2  w*  x^  4-  2  &[p    I  =  1  (k.2) 

j    i=l     X      K         i=l  lk 


and 


1   k  1    k+    k-  /,  _v 

I  w//  I  =  w£,  +  w£.    .  (^«3) 

In  this  case,  if  the  input  tolerance  is  specified  as  L,  the  inequality 


For  example,  the  minimization  of  objective  function 

R    n  k-1 

z  (  z  (wf+  +  w£~)  +  2  (at    +  a")) 
k=l  1=1     '  i=l  lk    lk 

guarantees  (U.2)  and  (i+.  3)  •   See  Section  5. 
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S(wf +  w|-)+kz'(a+k+a-k)<i  (k.k) 

L  =1  1=1 

for  the  k-th  gate  will  assure  that  the  input  tolerance  is  satisfied. 

The  restriction  on  the  maximum  number  of  inputs  for  each  gate, 
i.e.,  the  maximum  fan- ins,  can  also  be  considered.   Let  I  be  the 
specified  maximum  number  of  inputs  and  introduce  the  following 
variables  whose  values  are  0  or  1, 

n|<  1  (U.5)* 

and 

v..  <  1   , 

lk  — 

associated  with  w»  and  cc..  as  follows: 
*      lk 


k+    k-  ^  TT  k 


at.  +  a"    <  Uv.n  .  (U.6) 

ik    ik  —   ik 


The  fan-in  restriction  is  written  as 

n       k-1 

Z  u|  +  Z  v.,  <  I  (U.7) 

for  the  k-th  gate  (k  =  1,  2,  ...,  R).   u«  =0,  for  example,  means 

that  w^  and  w^~  are  both  0,  (see  (h£))   resulting  in  no  connection 

from  the  J?-th  external  input  variable  to  the  k-th  gate.  Accordingly 

(k.'j)   which  allows  at  most  I  of  p..  and  v   to  assume  1  guarantees  the 

-v      ik 


These  inequalities  are  not  needed  for  the  implicit  enumeration 
method. 


16 


satisfaction  of  the  given  fan- in  restriction. 

The  maximum  number  of  outputs  from  each  gate,  the  maximum  fan- 
outs,  can  also  be  treated  similarly. 

Next,  let  us  consider  the  restriction  on  the  number  of  levels 
of  a  network.   Let  us  assume  a  feed-forward  network  have  R  gates  and 
let  the  maximum  number  of  levels  be  J.   Then  there  arise  sets  of  in- 
terconnections which  are  not  permitted  because  of  the  level  restric- 
tion. For  example, 

a!2  *  °'  a23  *  °'  '••'  aJ,J+l  *  °'  aJ+l,J4£  *  °      {k'8) 

are  not  allowed  because  (J  +  l)  consecutive  interconnections  indicate 

existance  of  more  than  J  levels.   This  set  of  interconnections  can  be 

prohibited  by  using  the  zero-one  variables  v   defined  by  ( U. 5)  and  (h.6) 

ilc 

as  repeated  in  the  following, 


v..  <  1 
lk  — 


<k +  0L  s  Uvik . 


(U.9) 


The  inequality 


(k  =  2,   3,    ...,   R) 

(i  =  1,   2,    ...,    k  -  1). 


v12  +  v23+  ...   +vJ+1^<J  (U.10) 


assures  that  at  least  one  of  oc     ,  a     ,    ...,   a  is  0,  thus  de- 

li- c.i)  J+l,J+^ 

stroying  the  consecutive  interconnections  from  the  first  gate  to  the 
(J  +  2)-th  gate.  Consequently  a  procedure  to  keep  a  network  within  J 
levels  is*:   (l)  first  exhaust  all  sets  of  interconnections  which  have 


Alternatively,  we  can  employ  a  feed-next  network  instead  of  the 

feed-forward  network  as  a  basis,  so  that  the  level  problem  may 

[20] 

automatically  be  avoided.   See  the  reference 
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more  than  J  levels  and  (2)  add  all  the  corresponding  inequalities 
derived  in  a  similar  way  as  (1+.10). 

Other  types  of  restrictions  on  interconnections  can  usually  be 
incorporated.  For  example,  if  an  interconnection  a  is  prohibited 
for  some  reason,  we  can  simply  add 

at.  =  a"  =  0  .  (U.ll) 

lk    lk  v    ' 

The  conditional  restriction,  i.e.,  the  restriction  that  0:.,,  ,  =0 

l'k' 


if  a.,  -  0   holds,  is  realized  by 

lk 


ai'k«   +  QW  -U(l  "   Vik}  '  (^'12) 


where   V       is  the  zero-one  variable  defined  by  (k.y).      Namely,    if 

IK 

a._  4  0,  then  v.,  =  1  by  (U.9)  and  it  forces  a. ,,  .=  a~!~  ,  -  a"  ,  , 
lk  '   '      lk  i'k'    l'k*    l'k1 

to  0  by  (U.12). 

By  making  use  of  the  above  observations,  more  involved  conditions 
such  as  the  planarity  of  a  network  could  be  also  incorporated.   A  pro- 
cedure is  first  to  list  all  conditions  which  lead  to  the  non-planarity 
and  then  to  set  up  corresponding  inequalities  to  prevent  them.   However, 
if  R  and  n  increase,  the  number  of  inequalities  will  become  excessively 
large,  and  this  approach  would  be  impractical. 

One  way  to  avoid  this  difficulty  is:   (l)  solve  the  set  of  in- 
equalities without  adding  the  extra  inequalities  which  guarantees  the 
planarity.   (2)  If  a  solution  obtained  is  planar,  then  it  is  an 
optimal  solution,  otherwise  (3)  add  some  of  the  inequalities  from  the 
extra  inequalities  explained  above,  so  that  the  current  solution  may 
become  infeasible.   Then  solve  the  new  problem  resulted.   By  re- 
peating (3),  we  will  eventually  obtain  an  optimal  planar  solution. 
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When  we  solve  a  given  integer  linear  programming  problem  by  the 
implicit  enumeration  method,  restrictions  to  be  incorporated  need  not 
be  in  linear  inequalities.   Then  algorithms  to  test  the  planarity  of  a 
network  which  have  been  known  to  date  can  be  incorporated  without  being; 
written  in  linear  inequalities  in  the  integer  linear  programming  pro- 
blem of  Section  3«  And  a  planar  optimum  solution  will  be  obtained. 
This  approach  will  be  further  extended  and  explored  elsewhere. 
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5.   Procedures  to  design  optimum  networks. 

We  h^ve  so  far  discussed  only  the  constraints  of  the  integer 
linear  programming  problem  (l. l).   Now  the  objective  function  c"  y 
and  the  entire  procedure  for  deriving  an  optimum  network  will  be  pre- 
sented. 

First  consider  the  realization  of  a  feed-forward  network  which 
has  the  minimum  number  of  gates.   As  far  as  only  the  minimization  of 
the  number  of  gates  is  concerned,  any  expression  can  be  used  as  an 
objective  function  for  the  following  Procedure  I.   So  for  the  moment 
let  us  assume  that  we  can  have  an  objective  function  c   y  which  is  a 
linear  function  of  variables.   Concrete  expressions  for  c*  y  will  be 
discussed  later  in  this  section. 
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Procedure  I 

1)  Set  R  =  1  (if  we  know  a  lower  bound  Of  the  number  of  gates  required 
for  realizing  a  given  function  by  some  other  me,ans,  set  R  to  this  number). 

2)  Formulate  a  set  of  inequalities  for  a  R  gate  feed-forward 
network  and  possibly  inequalities  corresponding  to  the  restrictions 
imposed.   Solve  this  problem  with  an  objective  function  c  y  by  using 
an  appropriate  algorithm  of  integer  linear  programming.   If  this  has 
a  solution,  it  is  an  optimal  solution.   Otherwise  go  to  3)« 

3)  Increase  R  by  1  and  return  to  2). 

The  whole  procedure  will  terminate  in  a  finite  number  of  steps  if 
the  problem  is  feasible  for  some  R. 

An  integer  linear  programming  problem  in  the  above  procedure  is 
generally  a  mixed- integer  integer  linear  programming  problem.   However,  the 
problem  can  be  converted  into  an  all- integer  integer  linear  programming 
problem  by  restricting  all  variables  to  integers.   If  a  mixed- integer 
problem  has  a  solution,  so  does  the  corresponding  all- integer  problem, 
though  their  solutions  may  be  different.  In  some  cases, 
solving  the  all  integer  problem  may  be  easier  than  the  mixed  integer 
problem. 

Now  let  us  discuss  concrete  expressions  for  the  objective  function 
c  y,  when  we  solve  Procedure  I.   Since  Procedure  I  yields  a  network  of 
a  minimum  number  of  gates  for  any  expression  for  c"  y,  we  can  minimize 
(or  maximize)  other  parameter  of  a  network  at  the  same  time,  by  choos- 
ing an  appropriate  expression  for  c   y.   In  other  words,  Procedure  I 
will  yield  a  network  which  has  the  minimum  number  of  gates  and  an 
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optimized  parameter,  as  will  be  discussed  in  the  following. 
As  discussed  in  Section  k,    the  mimimization  of 


n      k-1 
^IwJ  +  .Eja.J  (5.1) 


as  the  objective  function  c  y  means  the  maximization  of  the  input 
tolerance  of  the  k-th  gate  defined  under  a  certain  definition  [1^].  As 
the  input  tolerance  is  a  maximum  permissible  deviation  of  gate  parameters 
such  as  weights  and  a  threshold,  it  means  the  maximization  of  the 
reliability  of  operation  of  a  gate.   Therefore  the  minimization  of  the 
sum  of  (5«l)  over  all  the  gates,  i.e. 


itJ^KsiKki)  (5-2) 


means  the  maximization  of  the  reliability  of  the  whole  network«(5-2)  can 
be  rewritten  in  a  linear  expression 


Ji ( Ji(wf +  vt">  +  iSi  Kk +  aik»         (5-3> 


since  with  the  minimization  of  (5»3)>  "the  relation  (k.3)    is  always  guaranteed. 
Note  that  (U.2)  also  holds. 

Under  another  assumption,  the  maximum  input  tolerance  for  the  k-th 

.  .   fPll 
gate  can  be  attained  by  minimizing1-   J 


V 
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rather  than  minimizing  (5*3) •   In  this  case, 


&K  +  Tk»  (5.m 


will  be  the  objective  function  to  minimize.   Also,  with  the  objective 
function  (5.k) ,  |t  |  =  T^  +  T~  obviously  hold. 

Instead  of  the  input  tolerance,  the  minimization  of  the  number  of 
interconnections  between  gates  can  be  taken  into  account  by  minimizing 
the  objective  function 


R  k-1 

,20  .Z.  v..  (5-5) 

k=2  i=l  lk 


where  v.,  are  the  zero-one  variables  defined  by  (J+.Q)*   Note  that  with 
lk 

the  minimization  of  (5.5), v=0  means  the  non-existence  of  the  inter- 

connection  (X..    while  v.  =1  means  the  existence, 
lk       lk 

The  number  of  connections  of  the  external  input  variables  can 
also  be  considered  by  using  the  zero-one  variables  u»  defined  by  (h. 5)  and 
(h.6) .      The  minimization  of 


R  k-1       R   n 
k=2  i=l  Vik  +  k=l  J&l  ^t  (5.6) 


will  yield  a  network  with  the  minimum  number  of  the  whole  interconnections 
including  those  from  external  inputs. 

An  alternative  approach  for  deriving  a  feed-forward  network  with  the 
minimum  number  of  gates  is  outlined  in  the  following. 
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Procedure  II 

First  prepare  a  sufficiently  large  number  of  gates  which  will 

guarantee  the  re aliz ability  of  a  given  function.   Then  introduce  new 

zero-one  variables  \    such  that 

k 

h§k+lKh  +  V  Su  \   (k=l,2,...,R-l).     (5.7) 

As  in  the  case  of  v.n  and  u  „,  if  there  is  at  least  one  inter- 

ik      £' 

connection  from  the  output  of  the  k-th  gate  to  any  other  gate,  then  X. 

ic 

assumes  1.   If  X.  =  0,  the  output  of  the  k-th  gate  is  not  connected  to 
others.   Thus  the  minimization  of 

R-l 

A  \  (5-8) 


will  result  in  the  minimum  number  of  gates  which  are  actually  used.   Note 
that  with  the  minimization  of  (5.8)  the  value  of  (5-8)  shows  the  number  of 
gates  actually  used  except  the  last  gate  which  always  exists  in  order  to 
produce  the  given  function  f . 

Although  this  procedure  needs  only  one  integer  linear  program,  the 
size  of  the  problem  is  relatively  larger  than  that  of  Procedure  I. 
Judging  from  the  fact  that  the  convergence  of  most  of  integer  linear 
programming  algorithms  become  progressively  slow  with  the  increase  of  the 
size  of  problem,  Procedure  I  may  be  preferred  at  present. 

An  upper  bound  of  the  number  of  gates  in  a  feed-forward  network  which 

T171  fell 
is  necessary  for  realizing  any  function  of  n  variables  is  known     u   J 

as  follows: 


2k 


n-1    n+1 
2  2   +  2  2   +1   for  odd  n  >  7 

2      +1      for  even  n  >  6  } 


which  may  be  used  as  the  number  of  gates  in  Procedure  II. 

An  advantage  of  Procedure  II  is  that  the  other  types  of  objective 
functions  such  as  discussed  in  conjunction  with  Procedure  I  can  be  used 
as  the  primary  objective  rather  than  the  secondary.   This  can  be  done 
simply  by  replacing  the  objective  function  (5*8)  by  an  objective  function 
such  as  (5.3)  or  (5«6).   This  procedure  may  yield  such  networks  as  one 
with  the  minimum  number  of  interconnections  but  with  the  number  of  gates 
which  may  not  be  minimum.   For  this  prupose,  therefore,  the  number  of 
gates  due  to  (5*9)  may  not  be  sufficient.   However,  to  authors'  knowledge, 
no  theory  is  available  about  such  number. 
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6.   Feed-forward  netwrok  of  majority  gates 

Instead  of  the  threshold  expression  which  has  been  used  in  the 
preceeding  sections,  the  majority  expressionL21]  is  used  often  for 
gate,   n  +  1  real  numbers 


O    1 


w  (6.1) 


are  associates  with  a  single  majority  gate.   A  majority  gate  with  n  variable 
inputs  operates  as  follows 


1  if  wo  +  Jx  .  t  <*>   >  1 


(6.2) 

n 


'  Wo  ^     .  W^  <  "I  . 


where  |    =  (|   ,  ...,    i        )    represents  the  j-th  input  vector  with  each 
component  | .   assuming  -1  or  1  instead  of  0  or  1.   The  k-th  majority 
gate  in  a  feed-forward  network  emits  the  output 


1  if  w^  +  PZ,  wVp^  +  :Z,  a.o[^   >  1 


o  '  4=1  VI  i=l  ik^i 

(6.3) 

"lifV  Ji^i')  +  iiiaikQiJ)<-^ 


where  cc   is  the  weight  of  interconnection  between  the  i-th  gate  and  the 
xk 

j-th  gate,  and  Q,.   is  the  output  value  of  the  i-th  gate  for  the  j-th 

input  vector,  which  is  also  1  or  -1.   Obviously  (6.2)  and  (6.3)  respectively 

correspond  to  (2.1)  and  (2.2)  of  the  threshold  expression.   Compared 
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with  the  threshold  expression  in  Section  2,  the  majority  expression 

can  describe  more  easily  some  properties  of  a  switching  gate.  As  far 

as  the  logical  operation  of  a  switching  gate  is  concerned,  both  expressions 

posess  no  essential  difference.  But  the  input  tolerance  of  a  majority 

gate  is  respresentedL1' -I  l^-LJ  -^y 


.1   |WQ  +  A  Vj,   +  ±h   QikQi  I  (6.U) 

n  i  k, 
*IolwJ 


The  set  of  inequalities  which  defines  a  feed-forward  network  with  R 
majority  gates  is  derived,  analogously  to  that  of  Section  3« 
Let 


7{p   =  a.a{^  (6.5) 


ik     ik  i 


and 


,(d) 

I 


P^  -  K  (6.6) 


'.    assumes  0  and  1  according  to  Q.   =  -1  and  1 
l  l 

For  the  k-th  majority  gate,  k  =  1,  2,  ...,  R-l, 


where  p.    assumes  0  and  1  according  to  Q.   =  -1  and  1  respectively. 


'o  4A^   +  ife'lk  ^  X  "  U  (l"PkJ) 


(6.7) 


-wk-  lw^(j)  -  ly^   >1  -UP[J), 
o  t=l   t  I  i=l  ik  —       k  ' 

(j  =  1,  2,  ...,  m)  , 
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y[p  <a.n    +  U  (I-P?J)) 
ik     —     ik  1 

a      <7{p  +  u  (i-p?^) 

ik  —     ik  i 


/ij"  <-a..    +  u  P. 
ik    —      ik  i 


(6.8) 


J')    ,  TT  JJ) 


"aik-  7ik    +  u  pi    '         (i  =  lf  2>   '">   k_l) 

( j   =  1,   2,    . . .,   m)    . 


For  the  last  majority  gate 


n  R    1 

E  1  (6-9) 

(j   =  1,   2,    ...,   m)   , 


7iR}   -aiR  +  U  (1-Piil)) 
aiR-  7iR}   +  U   ^"PP^ 


IE 
(J) 


r^;  <-a._  +  upH;  (6.io) 

lR     —        lR 


-aiR^7iR 


1 

UP  ^ 

) 

i 

, 

(i   = 

If 

2, 

•  •  •  f 

R- 

1) 

(j   = 

If 

2, 

•  •  •  f 

m) 

. 

Only  P.        are   zero-one  variables.      Other  variables  w„  ex.,,    7.,, 
l  V      ik'      ik  ' 

k  =  1,   2,    ...,   R,         £  =  1,   2,    ...,  n,    i  =  1,  2,    . ..,   k-1,    j  =  1,   2,    . . .,  m, 

are   all  real  numbers  which  are   actually  written  as 


k         k+         k- 

vl  =  vl     -  w^ 

aik  =  aik+  "  aik"  (6'11} 


y(d)  _7(J)+.r(j)- 

rik      '   'ik  7ik 
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in  order  to  keep  the  non-negativeness.   Variables  q)   do  not  appear 

explicitly  in  the  formulation.   The  set  of  inequalities  (6.7);  (6.8), 
(6.9)  and  (6.10)  together  completely  specify  a  feed-forward  network  of 
majority  gates.   The  rest  of  procedure  for  an  optimal  network  is  the  same 
as  that  for  threshold  gates  networks. 

In  this  case,  however,  the  restriction  on  the  input  tolerance 
discussed  in  Section  k   is 


rather  than  (1+.^-)  since  (6.k)    is  used  as  the  input  tolerance.   A  network 
with  a  maximum  input  tolerance  can  be  obtained  by  replacing  the  objective 
function  by 


Jl(wo+  +  wo~  +  ll^VP  +  Sl(at^ik))  (6-13) 


(it  is  proved  in  [21]  that  the  minimization  of  (6.13)  leads  to  the 
maximization  of  the  input  tolerance  of  all  gates.) 
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7.   Multiple  output  feed-forward  networks 

One  of  the  important  but  more  difficult  problems  is  the  synthesis 
of  a  multiple  output  network  in  which  more  than  one  function  is  simultaneously 
realized.   A  feed-forward  network  of  threshold  gates  generally  with 
multiple  outputs  is  displayed  in  Figure  2. 


R-l 


F;g.  2  A  Feed-Forward  Network  With  Multiple  Outputs. 


Each  output  from  a  gate  in  the  network  is  denoted  as  F  for  the 


k-th  gate,  where 


F   (^j))  =  Pn(j) 
k  k 


(7-1) 
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holds.   P^   is  defined  by  (3*3)  as  in  the  case  of  a  single  output 
network. 

When  given  S  functions  f  ,  f  ,  ...,  f   are  to  be  realized  with  the 
multiple  output  network  of  Fig.  2,    each  function  must  be  realized  by  one 
of  the  F  's.   (k  =  1,  2,  ...,  R)  If  f  is  realized  by  F 


f  (x^)  =   F,  (x^j)) 
a        b  (7.2) 

( j  =  1,  2,  .  ..,  ma) 


must  hold,  where  m   is  the  number  of  specified  vectors  for  f  . 

a  a 

Let  us  introduce  a  set  of  new  zero-one  variables 


*kt> 


k  =  1,  2,  ...,  R  (7-3) 

t  =  1,  2,   ...,  S  , 


where  f        =   1  if  the  output  of  the  k-th  gate  realizes  the  function  f  , 

and  t,  j_  =  0  otherwise.   Then 
kt 


R 

Z^^  =  1,   t  =  1,  2,    ...,  S  (7.U) 


holds  because   each  f ,    is  realized  by  exactly  one  output. 
Now,    if  $        =  1,    then 


PU)  =  f^d))  (7#5) 
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must  follow  from  (7.1)  and  (7-2).   This  is  realized  by  linear  inequalities: 


ft(?J))  <p^}  +u  (i-^kt) 

PkJ)  <ft(^j))  +u(1-tkt),  (7.6) 

(t  =  1,  2,  ...,  S) 
(k  =  1,  2,  ...,  R) 
(j  =  1,  2,  ...,  mt)  , 


Consequently  the  following  set  of  inequalities  characterizes  a  feed- 
forward network  with  R  gates  which  realizes  S  given  funstions.   The  rest 
of  the  network  can  he  treated  in  the  same  way  as  the  discussion  in 
Section  3- 

For  the  k-th  threshold  gate  for  all  k  =  1,  2,    ...,    R 


Znw^x(,d)  +  .S.p!^  -  T,  >  -  t/(l-p[d)) 

=1  I  £  i=l  lk     k  —    v   k 


(7.7) 


^kx[^    -  .Z-&[p   +  T   -  1  >  -  UPp\ 

^=1  t  I  i=lik     k    —     k  > 

(j  =  1,  2,    ...,   m)*  , 


S.P  <  a.,  +  UU-P^) 
ik  —  ik        1 


a.,  <  p[p  +  u(i-p:^) 

ik  —  ik        1  ' 


(7.8) 


(i  =  1,  2,    ...,    k-l) 
(j  =  1,  2,  ...,  m), 

i=5+i(pki    +  pki  ^UPk  >  (T>9) 

(J  =  1,  2,  ...,  m)  f 


*    j  runs  over  all  input  vectors  which  are  specified  for  at  least  one 
function  of  f  ,  fp,  ...,    fq. 

**   S  with  R  >  R  is  defined  to  be  0  to  eliminate  the  inequality  for  k  =  R. 
Rl 
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kiltkt=  (t  =  l,   2,....,   4  (7'10)* 


ft(^j))  <Pkj)  +  U(l^kt} 
Pfcj)   <  ft(?j))    +U(Hrkt), 


(7.11) 


(k . 

=  1, 

2, 

»  •  •  , 

R) 

(t  = 

=  1, 

2, 

•  •  •  > 

s) 

(J  - 

■  1, 

2, 

•  •  •  , 

m.)    . 

P,         and  ^n  ,    are   zero-one  variables.      Others  are  all  real  variables  which 
k  kt 

should  be   split   as,    for  example, 


k         kf         k-  ,_  _.s 


Of  course,  f,  (x   )  are  given  constants. 

A  procedure  for  a  minimum  gate  synthesis  is  similarly  performed  as 
in  Section  5  (of  course  a  secondary  objective  can  also  be  considered). 
Namely,  starting  from  R  =  S  (if  all  S  functions  are  distinct),  increase  R 
by  one  until  a  feasible  solution  is  found.   The  first  solution 
gives  an  optimal  solution. 

Other  considerations  in  Section  5  are  also  valid. 


This  may  be  given  by  two  inequalities. 
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8.   Synthesis  of  an  optimum  network  with  NOR  gates 

As  a  special  case  of  the  previous  discussion  a  single  output  NOR 

gate  network  will  be  considered.   This  is  important  for  the  practical 

computer  logical  design.   An  NAND  network  may  be  similarly  treated,  thaigh 

no  discussion  will  be  given.   The  use  of  more  than  one  type  of  gate,  such 

as  a  mixture  of  NOR  and  AND  gates  will  be  discussed  in  the  succeeding  paper 

[22].   Some  of  preliminary  computational  results  will  also  be  given  in  the 

next  section. 

The  Boolean  expression  of  an  NOR  gate  for  n  variables  x_ ,  x,.,  ....  x  is 

1'   2        n 


xn  <   x0  v  . . .  v  x  =  x.  .x_  . . .  x   .        (8.1) 
12  n    1  2      n 


In  other  words,  if  at  least  one  of  inputs  is  1  then  the  output  is  0, 
otherwise  (i.e.,  all  are  0)  the  output  is  1.   This  is  a  threshold 
function  with  weights 


w 


(-1,  -1,  ...,  -1)  (8.2) 


and  threshold  0. 

In  our  formulation  of  a  network  of  NOR  gates,  all  the  variables 
assume  only  1  or  0  which  respecitvely  represents  the  existence  or  non- 
existence of  interconnections  between  gates  and  from  external  variable  inputs. 
Our  computational  experience  which  will  be  outlined  in  the  next  section 
indicates  that  the  implicit  enumeration  method  is  more  suitable  to  our 
all-integer  linear  programming  problem  than  Gomory's  algorithms. 
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Let  us  define  variables  as  follows: 

]^ 
vp :   the  interconnection  to  the  k-th  gate  from  the  -L-th  external  variable 

k  k 

input,   v »  =  1  means  the  existence  of  the  interconnection  and  v.  =  0 

means  the  non-existence. 

CP.1  :  the  interconnection  from  the  i-th  gate  to  the  k-th  gate.  <P   =1 
lk  e       ik 

means  the  existence  of  the  interconnection  and  otherwise  the  non-existence. 

P   :the  output  value  of  the  k-th  gate  for  the  j-th  input  vector. 

p   :the  input  value  to  the  k-th  gate  from  the  i-th  gate  for  the  j-th  input 
ik 

vector. 

Similarly  to  a  general  threshold  gate,   the  k-th  gate,  k    =  i,   2,    . ..,   R-l, 
in  a  feed-forward  network  with  R  NCR  gates  can  be  described  by 


(8.3) 


£lVlXl  i=lPik     -        U    U  *k      } 


t  vkY^+  r  o^'  >  i  -  ijp^ 

mx-t  +  i=ipik  -       u\    > 

( j  =  1,  2,  . ..,  m) 


k  ( i) 
(8.3)   ionplies   that  if  at  least  one  of  v«x\   (the  input  value  from  the 

external  variable  input)  and  p. .7   is  one,  the  output  of  the  k-th  gate  is 

0,  and  otherwise  it  is  1. 

The  input  from  other  gates  satisfies  the  relation 


U)  m  p(d)  (8#1+) 

ik     1   ik  ' 


which  is  equivalently  written  in  the  following  set  of  inequalities 
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fcP(j)    _   qp       +  p[p   >  -i 

i  lk         lk     - 

■p{^    +  q>.t  -2p^   >  0  ,  (8.5) 

i  i&        lk    — 

(k  =  2,    3,    -..,    R) 
(i  =  1,   2,    ...,    k-1) 
( j    =1,    2,    ...,   m). 


(8.j)    corresponds  to   (3*13)    and.  (3»1^-).      Since  cp       and  p.        are  zero-one 

IK       IK. 

variables,  while  the  a.       and  p.   in  (3*13)  and  (3*1^)  are  generally  not, 

IK.       "XK. 

two  inequalities  are  sufficient  to  specify  the  behavior  of  (8.U). 

The  last  gate  is  treated  separately  since  the  output  is  specified  by 
the  given  function  f. 


k-1 


"  JlViJ)  "  i^^Oforf^^)-! 


(8.6) 


k-1 


.E-vJx^  +  .*ipty   :>  1    for  fU^)  =  0   , 

£=1  -c  t  i=l  lk  — 

j  =  1,  2,    ...,  m. 


All  inequalities  (8.3)  (8.5)  and  (8.6)  characterize  the  entire 

feed-forward  network.   The  tested  procedure  of  designing  an  optimal 

network  is  the  same  as  Procedure  I  of  Section  5  except  that  the 
number  of  interconnections 


R   n      k-1 

h^A  +  AV  (8-7) 


k 


is  used  as  the  objective  function  since  it  is  important  in  practice. 
Note  that  the  expression  for  the  number  of  interconnections  needs  more 
involved  argument  (see  (5-6))  in  the  case  of  general  threshold  gate 
network. 
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When  we  solved  this  all-integer  integer  linear  programming  with  the 
implicit  enumeration  method,  we  observed  a  very  explicit  general  tendency 
that  the  smaller  the  solution  space,  the  faster  the  termination  of  the 
computation.   Therefore,  efforts  were  made  to  reduce  the  size  of  the 
actual  solution  space  to  be  calculated  without  losing  the  generality  of 
the  problem.   In  other  words  we  added  constraints  so  that  solutions  which 
are  obviously  not  optimum  or  those  which  can  be  easily  obtained  from  other 
solutions  by  permuting  variables  are  suppressed.   Important  constraints 
among  them  will  be  briefly  discussed  in  the  following  without  proof. 

(l)   If  the  k-th  gate  and  the  last  gate  are  connected,  then  It  is 
proved  that  other  outputs  from  the  k-th  gate  have  no  contribution  to  the 
output  of  the  last  gate.  (Fig.  3)   In  other  words,  if 


Fig.  3«   Redundant  Connections 


cpn  _  =  1,  then  we  can  automatically  set 
kR 


Wl  =  W  =  '••  =CpkK-l  =  °  '  (8'8) 
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(2)   If  the  i-th  gate  is  connected  to  the  k-th  gate  (no  other  output 
connection  from  the  i-th  gate)  and  the  k-th  gate  has  only  one  input 
connection  as  shown  in  Fig.  k, 


1  "ik     ^k 


o^-o 


Fig.  h     Cascaded  Connection 

then  the  input  value  to  the  i-th  gate  and  the  output  from  the  k-th  gate 
have  the  same  value.   Thus  both  gates  could  be  eliminated  from  the  network. 

(3)   Suppose  that  the  i-th  gate  is  connected  to  the  k-th  gate  (i  <  k) 
but  no  other  output  from  the  i-th  gate  exists.   Then  it  is  unnecessary  that 
both  gates  to  have  inputs  from  the  same  external  variables. 

(h)      Considering  constraint  (l),  it  is  possible  without  loss  of 
generality  to  impose  an  ordering  on  the  connection  to  the  last  gate  such  that 


VlE^VsR^-"  ^"lE  (8>9) 


(5)   Some  of  geometrical  symmetries  were  also  taken  into  consideration. 
For  example,  if  the  (R-2)-nd  and  (R-l)-st  gates  are  connected 


R-3 


R-2 

o     g>c^ 


Fig.  5-   Symmetry  Connection 
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to  the  last  gate  and  ^   _  _  .  =0  (this  is  in  fact  guaranteed  by 
constraint  (l))  then  the  (R-l)-st  and  (R-2)-nd  gates  are  symmetric  to  the 
rest  of  preceeding  gates.   This  justifies  a  new  ordering  such  that 


cp        >  cp      n  (8.10) 

Tt-3,  R-l  -  R-3,  R-2  . 


This  means  that  the  (R-3)rd  gate  is  connected  to  the  (R-l)st  gate  before 
it  is  connected  to  the  (R-2)nd  gate.  This  type  of  symmetry  was  extensively 
studied  with  other  gates  also.   However  the  details  are  omitted  here. 

(6)  Other  types  of  constraints  were  also  incorporated  such  that  each 
gate  has  at  least  one  input  connected  and  also  at  least  one  output  when 

all  R  gates  are  assumed  to  be  necessary. 

[31 
All  these  constraints  can  be  expressed  in  inequalities 

These  inequalities  in  addition  to  the  original  set  of  inequalities 

which  characterize  a  feed-forward  network  has  significantly  reduced  the 

computation  time.   Some  results  will  be  reported  in  the  next  section. 
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9.   Computational  experience  of  designing  optimum  networks  with  NCR  gates 

Integer  linear  programming  for  designing  an  optimum  feed-forward 
network  of  NOR  gates  in  Section  8  was  Implemented  on  the  IBM  360/75 
and  the  ILLIAC  II  of  the  University  of  Illinois.   All  three  variable 
Boolean  functions  are  synthesized  and  compared  with  Hellerman's  result 


by  the  exhaustive  method 


[13] 


Table  1  shows  the  size  of  problem  for  each  R  excluding  additional 
inequalities  discussed  in  the  second  half  of  Section  8. 


Table  1  Sizes  of  Integer  Linear  Programming  Problems 


R 

Matrix 

size 

Coefficients 

Constraints 

Variables 

Total  No. 

Non-Zero  ■ 

°]0  Non-Zero 

2 

,ko 

23 

96O 

1U0 

11+.58 

3 

88 

52 

k66h 

332 

7.12 

1+ 

152 

90 

13832 

596 

U.31 

5 

232 

137 

32016 

932 

2.91 

6 

328 

193 

63632 

13^0 

2.11 

7 

kko 

258 

113960 

1820 

1.60 

These  numbers  fluctutate  slightly  with  different  Boolean  functions. 
These  numbers  are  for  the  function  which  is  identically  1. 
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The  majority  of  published  reports  on  computational  efficiency  of 
integer  linear  programming  algorithms  have  the  number  of  variables  greater 
than  the  number  of  inequalities.   Our  problem  is  opposite.   The  number 
of  inequalities  is  far  greater  than  the  number  of  variables.  As  a  result 
when  the  implicit  enumeration  method  is  applied,  it  is  difficult  to  find 
a  feasible  solution  of  this  problem  but  once  found,  it  is  usually  close 
to  the  optimal  solution,  whereas  most  of  the  problems  published  in  the 
literature  of  operations  research  are  easy  to  get  the  first  feasible 
solution  but  take  long  computation  time  to  improve  it  until  an  optimal 
solution  is  found. 

The  non-zero  coefficients  are  very  few  as  seen  from  Table  1.   This  war 
fully  utilized  in  programming  to  speed  up  the  computation. 

The  number  of  additional  inequalities  used  for  the  derivation  of  all 
optimal  networks  for  3  variable  Boolean  functions  is  shown  in  Table  2. 
Some  of  them  are  very  effective  in  reducing  the  computation  time  but  some 
others  are  not  too  effective. 


R 

no.  of  additional  ineq. 

h 

17 

5 

33 

6 

87 

7 

276 

Table  2.   Number  of  Additional  Inequalities 
Gomory's  all-integer  integer  linear  programming  algorithm'- -^ J  was 
first  used.*  The  detailed  result  will  be  found  in  [2].   This  approach 
was  not  satisfactory  at  least  for  our  problem,  nevertheless  we  solved 


This  formulation  needs  such  a  set  of  inequalities  as 

-I  <  1,  *tt  <  1,  p$  <  1,  p[d)  <  1 

which  are  not  included  in  Table  1.   For  the  sake  of  computational 
efficiency,  a  slightly  modified  formulation  was  actually  used  [2]. 


considerably  larger  integer  programm_ng  problems  than  previously  reported 
attempts.   Formulations  for  R  =  3  networks  were  solved  within  10  seconds 
on  the  average  using  the  computer  ILLIAC  II.  However,  computation  time 
for  R  =  k   networks  were  very  erratic.   Some  of  them  were  solved  within 
20  minutes  but  others  were  not. 

Discarding  Gomory's  method,  the  implicit  enumeration  was  implemented 
on  the  IBM  360/75  with  FORTRAN  IV.  Several  modifications  and  improvements 
were  also  incorporated.   As  seen  in  Tables  1  and  2,  the  number  of  inequalities 
increases  as  R  increases.   But  we  made  modifications  such  that  computation 
time  is  least  affected  by  the  number  of  inequalities.   The  detail  will  be 
included  in  future  papers 

Optimal  NOR  networks  for  all  the  3  variable  Boolean  functions  were 
obtained  within  110  minutes,  by  solving  our  integer  program  as  a  general 
integer  program  [15].   This  may  be  favorably  compared  with  Hellerman's 
exhaustive  method  in  which  he  had  solved  the  same  problem  consuming  about 
26  hours  of  IBM  7090  computer  time.   The  effect  of  additional  inequalities 
was  remarkable.   Some  problems  for  R  =  6  formulation  was  speeded  up  in 
the  factor  of  6.5  in  computation  time. 

Considerable  speed-up,  however,  was  obtained  by  a  more  sophisticated 
modification  of  the  implicit  enumeration  method  by  taking  into  account  the 
physical  structure  of  a  network  [3].  For  example,  computation  time  for  a 
3  variable  function  of  6  NOR  gates  was  reduced  to  3  seconds  by  this  modifica- 
tion from  33  seconds  by  the  above  general  method. 

We  are  currently  trying  to  improve  the  computation  time.   And  the 
above  figures  should  not  be  regarded  as  the  maximum  attainable  ones. 
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Advantage  of  the  implicit  enumeration  method  over  Gomory's  algorithm 
is  not  only  computational  speed  but  also  ease  of  non-optimal  solutions.   In 
other  words,  whenever  we  stop  computation,  the  best  solution  obtained  by 
then  is  likely  close  to  the  optimum  one. 

This  result  not  only  confirms  the  computational  feasibility  of  the 

integer  programming  approach  in  logical  design  but  also  should  encourage 
its  adoption  in  other  fields. 
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10.   Conclusion 

A  feed-forward  network  of  threshold  gates  was  formulated  as  an  integer 
linear  programming  problem.   A  specific  problem,  a  NOR  gate  network,  was 
tested  by  existing  integer  linear  programming  algorithms,  and  shown  to  be 
computationally  feasible.   In  the  light  of  these  results  it  is  concluded 
that  the  integer  linear  programming  formulation  provides  us  with  a  new 
tool  for  optimization  problems  in  logical  design  which  otherwise  seem 
impossible  to  solve  theoretically.   The  approach  has  many  advantages  in 
incorporating  various  restrictions  on  a  network,  choosing  various  design 
objectives  and  treating  a  multiple  output  network. 

A  generalization  of  the  approach  as  well  as  the  formulation  of  a 
design  method  of  a  sequential  network  will  be  included  in  the  succeeding 
paper  [22]. 
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